Custom caching

ABSTRACT

Methods and systems are presented for custom caching. Application threads define caches. The caches may be accessed through multiple index keys, which are mapped to multiple application thread-defined keys. Methods provide for the each index key and each application thread-defined key to be symmetrical. The index keys are used for loading data from one or more data sources into the cache stores on behalf of the application threads. Application threads access the data from the cache store by providing references to the caches and the application-supplied keys. Some data associated with some caches may be shared from the cache store by multiple application threads. Additionally, some caches are exclusively accessed by specific application threads.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/633,782, filed on Feb. 27, 2015, now U.S. Pat. No. 9,547,601, issuedJan. 17, 2017; which is a continuation of U.S. patent application Ser.No. 13/619,270, filed on Sep. 14, 2012, now U.S. Pat. No. 8,996,534,issued Mar. 31, 2015; which is continuation of U.S. patent applicationSer. No. 13/026,909, filed on Feb. 14, 2011, now U.S. Pat. No.8,301,590, issued Oct. 30, 2012; which is a continuation of U.S. patentapplication Ser. No. 11/965,458, filed on Dec. 27, 2007, now U.S. Pat.No. 7,890,537, issued Feb. 15, 2011; which is a continuation of U.S.patent application Ser. No. 11/007,061, filed on Dec. 8, 2004, now U.S.Pat. No. 7,406,464, issued Jul. 29, 2008, which claims priority to andthe benefit of U.S. Provisional Patent Application Ser. No. 60/528,053filed on Dec. 8, 2003; U.S. Provisional Patent Application Ser. No.60/528,237 filed on Dec. 8, 2003; and U.S. Provisional PatentApplication Ser. No. 60/528,238 filed on Dec. 8, 2003; the disclosuresof all of these applications and patents are incorporated by referenceherein.

FIELD OF THE INVENTION

The invention relates generally to data management and more particularlyto cache customization.

BACKGROUND OF THE INVENTION

Technological advancements have led to more sophisticated and powerfuldata management systems. Data processing in such systems requireefficient handling a large number of data transactions (e.g. data reads,writes, etc.).

The advent of the Internet and the World-Wide Web (WWW) combined withthe development of network-based commerce system applications haveenabled companies to transact business globally in real-time. Suchnetwork-based commerce systems may process large amounts of datatransactions from geographically dispersed users. The users may entertransactional requests over communications lines that result in thepersistant storage of data in a master information source (data store).In several cases, back-end systems (e.g., database servers, etc.)support the data store and interact with other systems or services tomap the data between a persistent or relational form into an object formthat is more readily utilized by software applications.

Furthermore, software developers typically expend a large amount of timeand energy in managing access related to the data store and in managingmemory usage within their applications, which is used to manipulate dataacquired from the data store within the scope of their applications. Onetool that developers rely on for improving processing throughput oftheir applications is a cache manager or cache service. However, cacheservices are typically not customizable and are not extensible by thedevelopers within their applications. This is so, because cachingservices are generally implemented as black boxes having a defined setof generalized and static services or features. As a result, data storeaccess and memory management are tasks that developers still largelyhave to account for within their individual applications. These tasksare time consuming and in some cases not optimal because some developersmay not be as adept as other developers in handling these tasks.

SUMMARY OF THE INVENTION

In various embodiments, extensible caching services and techniques arepresented. An application's access to a data store is transparent to theapplication. An application can customize, extend, and instantiate itsown caching environment. The caching techniques or services utilizedwithin the caching environment of the application provide seamless andtransparent access to data residing within the data store while managingthe applications' memory usage in optimized manners.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a method for custom caching, according to anexample embodiment.

FIG. 2 is a diagram of a method for key management within a customcache, according to an example embodiment.

FIG. 3 is a diagram of another method for custom caching, according toan example embodiment.

FIG. 4 is a diagram of a custom cache system, according to an exampleembodiment.

FIG. 5 is a diagram of another custom cache system, according to anexample embodiment.

FIG. 6 is a diagram of example network-based commerce system or facilityhaving the methods or systems of embodiments associated with theinvention.

FIG. 7 is a database diagram illustrating an example database, which atleast partially implements and supports the network-based commercesystem of FIG. 6, according to an example embodiment.

FIG. 8 is a diagram of an example listing table associated with thedatabase of FIG. 7, according to an example embodiment.

FIG. 9 is a diagram of an example users table associated with thelisting table of FIG. 8 and the database of FIG. 7, according to anexample embodiment.

FIG. 10 is a diagrammatic representation of an example machine in theexample form of a computer system 1000 within which a set or sequence ofinstructions, for causing the machine to perform any one of themethodologies discussed herein, may be executed, according to an exampleembodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a diagram of a method 100 for custom caching, according to anexample embodiment. The method 100 (hereinafter “custom-cache service”)is implemented in a machine-accessible or readable medium and isaccessible over a network. The network may be wired, wireless, or acombination of wired and wireless. The processing of the custom-cacheservice provides customized caching to applications. The applicationsare written by developers using an Application Programming Interface(API) that accesses the processing of the custom-caching service.

In an embodiment, the API which interfaces applications to thecustom-caching service is implemented in an Object-Oriented (OO)fashion. Accordingly, application source code includes class files thatprovide the API and that reference different aspects and features of thecustom-caching service through objects which are defined in classes andeach class has a variety of methods which access the features. Theobjects abstract and encapsulate different aspects associated withcaching. The API provides application-specific environments which cancustom control and access different features associated with caching;these features and control have traditionally been unavailable todevelopers of applications.

Each application processes on a machine with its own processing space ormemory heap. A single application may have a plurality of threads thatshare the same heap. Each thread may independently interact with thecustom-caching service. Alternatively, collections of threads mayjointly interact with the custom-caching service. Moreover, thecustom-caching service may synchronize data associated with differentapplications having different heaps.

The custom-cache service provides instances of caches to the applicationthreads. As used herein a cache is intended to include an abstracted andencapsulated version of a cache store. The cache store includes memorylocations within heaps of the application threads. The applicationthreads define their caches and access them through the API whichinteracts with the custom-cache service. In this manner, control overthe actual cache stores is transparent to the application threads andmanaged by the custom-cache service. Accordingly, memory management ofan application thread's heap can be delegated to the custom-cacheservice while access control is provided through higher-level interfacesassociated with the API, which is directly accessible to the applicationthreads.

In addition to provide customized caching, embodiments of thecustom-caching service reduces the load on data stores. This is so,because transactions to the data store are substantially reduced bypre-acquiring data frequently used by threads in single transactions tothe data store and by sharing frequently used data across multiplethreads through a single cache store. As a result, threads that performnumerous data store transactions can reduce the number of theirtransactions to the data store. Moreover, because data consumed by thethreads are immediately available when used from the cache store, theprocessing throughput of the threads can be substantially improved.

The custom-caching service may support two types of caches: a globalcache and a thread local cache. The threads have control through callsto the API to define the type of cache that best suits the programmingenvironment of the threads. A global cache includes data within itscache store that may be shared across multiple threads of anapplication. Data of a global cache is typically immutable; meaning thatthe data is typically static to semi-static in state and refreshed atintervals based on configurable policies enforced by the custom-cachingservice. A thread local cache includes data within its cache store thatis consumed and accessed only by a specific thread that creates thethread local cache. Furthermore, a single thread may utilize both aglobal cache and a thread local cache. Data within a thread local cachemay be changed, dynamic, and refreshed based on instructions issued bythe thread that created the thread local cache.

Initially, an application uses a call selected from the API to declarean instance of a cache within the programming logic of the application.The program also uses a variety of other calls within its programminglogic to drive the processing of the custom-caching service. Theprocessing of the custom-caching service is activated and executedduring the execution of the application.

With this example context, the processing of the custom-caching serviceis now described with respect to FIG. 1. Accordingly, at 110, thecustom-caching service receives a request from the application to createa cache. Initially, the cache is null having no data. At 120, thecustom-caching service receives configuration information from theapplication for the cache. In an embodiment, the configurationinformation is received when the cache is created for the application;allowing the constructor of the cache to accept optional parameters fromthe application which can include the configuration information. Theconfiguration information includes a pointer to a key manager and thekey manager has one or more key definitions.

Initially, before supplying configuration information, the applicationdefines a key manger that includes one or more key definitions. The keydefinitions may be viewed as an application supplied or pre-configuredservice or object for key types, such as an identifier used within adata object to uniquely identify that data object. For example, one keytype might be a social security number (SSN) field name of a data objectthat is a customer record from on online merchant's customer database.Each key definition accepts a value associated with a piece of dataassigned to a specific key type. For example, a particular keydefinition may accept the following parameter 555-55-5555, where theparticular key definition is associated with a type SSN. In response tothe value supplied to the key definitions, the key definitions generatea unique key. The key manager also knows how to accept a data object,parse it, and call each unique key definition associated with thedifferent key types included within that data object. In someembodiments, the key manager is a generic object that is simplyconfigured within the application by accessing the API to return apointer to an instance of a generic key manager.

The configuration information may include a variety of information, suchas an identifier for a data source and a mechanism, such as a searchquery, for acquiring a subset of data from the data source. In anembodiment, the custom-caching service may provide a library of genericor known data sources having generic instances of key managers. In theseembodiments, the application can supply configuration information byidentifying through the API the generic data sources and/or the genericinstances of key managers. Thus, the application may directly orindirectly supply the configuration information acquired, at 120, by thecustom-caching service.

At 130, the custom-caching service loads or populates portions of datainto a cache store associated with the newly created cache. Again, thecache store is not directly accessible to the application. Theapplication uses calls within its programming logic to define,configure, and access a cache. The custom-caching service manages andmaintains the data and the cache store, operations to set, get, write,flush, and/or update the data are provided through the API to theapplication for instructing the custom-caching service to act on thecache store on the application's behalf.

During a load, the custom-caching service passes a data object or aportion of data to each key definition associated with that objectthrough the key manager that was supplied by the application. The keydefinitions for each key type parse the data object and extract thenecessary values for form their respective keys. This results in one ormore keys from the key definitions. These keys are used within an indexof the cache store. Each index entry includes a key and a pointer towhere the data object physically resides within the cache store.

The portions of the data may be loaded by the custom-caching serviceinto the cache store in a variety of manners. In an embodiment, at 131,the custom-caching service may have received via configurationinformation an identifier to a data store (e.g., database, collection ofdata bases organized as a data warehouse, directories, electronic files,or various combinations of the any of these) and a query, which definesthe portions of data to be loaded into the cache store.

In an embodiment, at 132, the custom-caching service may be instructedby the application to load all portions of the data into the cachestore. This is useful when the cache is a global cache and where thevolume associated with the data is not excessive and is manageablewithin the heap or cache store.

In another embodiment, the custom-caching service may be instructed bythe application to load the portions directly at a given point in timeduring the execution of the application. This is useful when the cacheis a thread local cache and where the volume of the data is perhapsexcessive and it is desired to have only a portion in memory at any onetime or where the data is going to be changed by the application. It mayalso be that the application wants be sure that when it instructs thecustom-caching service to load a specific portion of the data that theportion is going to be as up-to-date as is possible.

Additionally, at 133, some portions of the data may be acquired by thecustom-caching service from a plurality of perhaps disparate datasources. For example, the application may load data from one databasefor some data types being consumed by the application and simultaneouslyadd data from one or more different databases for other data types beingconsumed.

Control of data types, data sources, and loading conditions of datawhich is loaded into the cache store is delegated to the applicationthrough calls to the API within the programming logic of theapplication. Some control is defined by the operations taken by theapplication, such as when the application uses an API call or operationassociated with a global cache, implying that the all the data is to beimmediately loaded by the custom-caching service into the cache store.Other control is more explicitly exercised by the application, such aswhen the application accesses calls or operations associated with athread local cache, indicating that the application is assuming directcontrol over when data is loaded and/or modified within the cache store.

In an embodiment, at 134, the custom-caching service may iterate theload processing in response to methods associated with the cache. Thesemethods are accessed via the API during the execution of theapplication. Again, this scenario is typically indicative of a threadlocal cache where the methods are calls or operations of the API andwhere the application selectively desires to perform loads duringcertain execution states of the application, but may also be used toload a global cache as data is demanded, assuming the data in the datasource (e.g., data store, database, etc.) is typically static orsemi-static.

At 140, the custom-caching service returns to the application areference to the cache. The application is now free to perform a varietyof operations on the data loaded within the cache store via accessingmethods associated with the reference to the cache. To do this, theapplication will call specific key definitions within its programminglogic and pass values for specific key types. The key definitions returna key that is symmetric with a key produced previously during the loadby the custom-caching service. The combined application-generated keyand the reference to the cache provide enough information to thecustom-caching service for locating the appropriate cache storeassociated with the cache reference and for locating the appropriatedata within the cache store. Again, the application-supplied keyprovides a direct location within the cache store index and thatlocation includes a pointer to the physical location of the data desiredby the application.

Once the portions of the data are indexed and loaded within the cachestore, the custom-caching service may also, at 141, refresh, flush, orupdate the data or selective portions of the data within the cache storeor to a data store, such as when an update is being performed. A varietyof mechanisms may drive the custom-caching service to perform thesetypes of operations such as when intervals for performing theseoperations are configured into the custom-caching service or definedwithin a policy that is dynamically enforced by the custom-cachingservice. Other techniques may include performing the operations afterreceiving a specific instruction from the application, performing theoperations after receiving an event trigger raised by a data store whena field or table of the data store is modified, and/or performing theoperations after receiving an event raised by a global administratorservice that communicates with the custom-caching service.

Some of the processing described above prior to 140 may have or have notoccurred before step 140. For example, data may not have been loaded tothe cache store and index keys to portions of the data may not have beenestablished. In this manner, the number and arrangement of theprocessing blocks of FIG. 1 are not intended to impart any particularsequence.

As stated above and as indicated, at 150, in some embodiments where thecache is a global cache the custom-caching service may share the cachestore and the data loaded therein with other threads of the application.The threads acquire the reference to the cache from the application orparent thread and may access the cache store through the reference asdesired.

In still another embodiment, also at 150, the custom-caching service maysynchronize loaded data within the cache store associated with theapplication's heap with other loaded data within other cache storesassociated with different applications having different programmingheaps. This may be useful when the same data acquired from the same datasource is loaded within two different cache stores and there is a desireto maintain synchronicity of the data state.

FIG. 2 is a diagram of a method 200 for key management within a customcache, according to an example embodiment. The method 200 (hereinafter“key service”) is implemented within a machine-accessible and readablemedium and is accessible over a network. The network may be wired,wireless, or a combination of wired and wireless. The key serviceperforms a variety of techniques for managing keys associated with anapplication and a custom-caching service, such as the custom-cachingservice represented as the method 100 of FIG. 1. In an embodiment, thekey service is implemented within the custom-caching service as an OOcomponent or a sub-service associated therewith.

As will be demonstrated below, the key service provides novel featuresthat permit data stored within cache stores to be uniquely associated,grouped, indexed, shared, and retrieved from the cache store. In thismanner, the key service extends the functionality of the custom-cachingservice of FIG. 1 providing added flexibility to applications. Again,some features presented herein and below may be accessed via the API ofthe custom-caching service and other features may be internal to thecustom-caching service.

At 210, the key service acquires a key manager for a particular cache.Acquisition may be achieved via a pointer supplied by the application tothe key service. The key manager includes a variety of key definitions.The key definitions provide symmetrical keys between the application andthe key service. Each key definition operates on a particular key typeassociated with a data object. A key definition returns a symmetricversion of a key in response to receive a particular value for aparticular key type. The keys returned from the key definition providename spacing that delineates between different key types for the samedata object type, and delineates between different key types fordifferent data object types, so that keys with equivalent data, whichpertain to different values and/or objects, can be delineated betweenthe index. This is typically achieved through different actual classtypes for the key types, which are used during index lookup. The keydefinition may be supplied the direct data, or may parse the data froman instance of an object that it relates to, in order to create a keyobject.

At 220, the key service populates a cache store by using unique keydefinitions acquired from the key manager or processed through the keymanager. Moreover, for each portion of data received in a data objectfor loading, an appropriate key definition is processed. In anembodiment, at 221, the key service may initiate the populationprocessing in response to an instruction received from the application.In this manner, the application may perform on demand loading, such aswhen the application is using a cache associated with a thread localcache, or loading a global cache based on demand for data.

At 230, a key is received from an appropriate key definition for eachportion of the data during the population, at 220. The key is consumedby the key service for acquiring a specific unique location within anindex associated with the cache store. At 240, that key is stored in theindex along with a reference to the data associated with the portion ofthe data that is being processed. The reference is stored within anentry value of the index; the location of the entry value within theindex is acquired via the key.

The processing at 220, 230, and 240 is iterated, at 250, for identifiedkey types and key definitions associated with the data. Thus a singlepiece of data or single data object may have a plurality of keys andentries housed in the index of the cache store and each of those keysinclude the reference to the physical location of the data object withinthe cache store.

At 241, the storing of the reference may be modified to accommodatecollection keys. A collection key is provided by a special type of a keydefinition. A collection key is a logical grouping of portions of thedata associated with same key type. For example, suppose that theapplication defined via a key definition that each set of data returnedfrom the announcement data store where announcements associated with thesame ISP be grouped into buckets, and the buckets associated with acollection key. In the example, maybe 100 announcement records aregrouped into a first bucket with a first collection key because they allhave an ISP value of “AOL” and another 10 customer records are groupedinto a second bucket having a second collection key because they allhave an ISP value of “Roadrunner.” The groupings that define acollection key may be established by the application through the a APIusing key definitions, operations, or calls. One appreciates, that thisis a powerful feature since it can, among other things, preorder datawithin the cache and group data into buckets, such that the order andbuckets can be later accessed without having to perform an additionaloperation with the application when the order or buckets are needed.

To accommodate this novel collection key there may be multiple portionsof the data that have the same key to the same entry value within theindex. The key service knows that the key definition used to acquire thekeys was a collection key definition. Therefore, the key service willhandle the situation during subsequent iterations of the key servicewhen it encounters a portion of data that indexes to an alreadypopulated entry value which already has a reference included therein.Thus, at 241, the key service will acquire the pre-existing reference,which will typically be a list type structure, and add the pointer ofthe new data object to the list. If an entry does not already exist fora collection key entry, then it will create a new list structure and addthe new data object to this list and put the collection key and listinto the cache store. Thus, the duplicate references are populated as aplurality of entries within the data structure. In this manner, theapplication has logically grouped portions of its data loaded within thecache store into collections for future ease of reference and directaccess.

In some embodiments, an application may try to access data not yetloaded within the cache store. In these cases, the key service mayreturn a null reference, indicating that an entry does not exist, orgenerate an exception. At this point, the application can attempt toacquire the data from a data store and then load the data into the cachestore through methods associated with the cache reference and thenproperly acquire the data. Additionally, if the data is not found in thedata store, then the application can put a marker object into the cachefor this particular key, to indicate that an object does not exist forthis key, so that subsequent thread requests will not bother calling thedata store to look for the data that does not exist, thus reducing loadon the data store. Lazy loading may occur with both global cache typesand thread local cache types.

Once data is loaded into the cache store, the application can retrieveselective portions of the loaded data by passing the reference to thecache and a selective key within an API call or operation to acquire thedata from the cache store. This is achieved by using the selective keyto locate the entry value within the index of the cache store, acquiringthe populated reference, and retrieving the selective portions of theloaded data from a physical location within the cache store identifiedby the reference.

In an embodiment, the cache may also receive from a different thread ofthe application a specific key into the index along with a reference tothe cache. In response to this, the key service returns the dataacquired by traversing the reference located at the entry value locationwithin the index pointed to by the specific key, which was supplied bythe different thread.

In yet another embodiment, at 260, a write or update operation may besubsequently received from either the application or a different threadof the application. The operations include a reference to the cache, anew piece of data, and a key. The write operation, at 261, acquires theappropriate index entry value location pointed to by the key, andacquires the reference to the location of the data within the cachestore, and replaces the data within the cache store with a new piece ofdata. This update is realized by all key entries for this object in theindex, as they hold a reference to a cache entry holder object, whichhas a reference to the data object, which is what is actually updated;thus, updating the data object reference for one key cache entry updatesit for all key entries for that object. A write occurs with a cachereference associated with a thread local cache. An update operationacquires the matching location in the same manner, but uses otherservices to write the matching data in the cache store to one or moredata sources or data stores, or is handled externally at the discretionof the application.

FIG. 3 is a diagram of another method 300 for custom caching, accordingto an example embodiment. The method 300 (hereinafter “thread localcache service”) is implemented in a machine-accessible and readablemedium and is accessible over a network. The network may be wired,wireless, or a combination of wired and wireless. In an embodiment, thethread local cache service represents a sub-service or an OO componentof the custom-caching service represented by the method 100 of FIG. 1.Additionally, the thread local cache service may also integrate andinterface with the key service represented by the method 200 of FIG. 2.

At 310, the thread local cache service, receives a request from a threadto obtain a reference to its thread local cache. The thread local cacheinstance is created for a thread if one does not already exist withinthe thread local cache service for it. The thread local cache instancereceives an instruction from the application thread to begin cachingdata objects that the thread encounters during its processing. Thisinstructs the internals of the thread local cache instance to enableoperations or methods on this thread cache instance to read and putobjects in the thread local cache's cache store. It is appreciated thatother techniques are permissible to obtain an instance of a thread localcache. For instance, in an OO implementation, the cache may be utilize aspecific thread local cache constructor, may pass specific parameters oridentifiers to a generic cache constructor, or simply issue a begin orsome other type of operation that identifies a generic cache as beingassociated with a thread local cache.

At 320, the thread local cache acquires a key manager having keydefinitions from an application thread as it visits the objects thathandle the loading of specific object types into the thread local cache.Techniques for generating and providing key definitions were discussedin detail with respect to the key service of FIG. 2. However, in thecase of the thread local cache, the key manager or a specific keydefinition are received by the thread local cache when the thread wantsput an object into the thread local cache. Thus, key managers and theirkey definitions are discovered during runtime as objects are put into athread local cache.

Additionally, in some embodiments, at 321, some key definitions receivedfrom the application thread by the thread local cache may be fordifferent object types. This means that multiple types of data objectsfrom multiple data sources may be defined within the thread's threadlocal cache; such the key definitions are significantly different fromone another and are specific to the different object types.

At 330, the thread instructs the thread local cache to selectively loadportions of data to the cache store via API calls on thread local cache.At this point, the thread local cache service interactively and in apiecemeal fashion, as directed by the thread, assembles the desired datainto the cache store using the key manager and its key definitions (inmanners discussed above) and in the process constructs an index into thecache store for acquiring the desired data. In addition, at 331, the keydefinitions supplied at 320 may define a number of the key definitionsas collection keys. Examples of a collection keys were provided abovewith respect to the key service of FIG. 2.

At some point during the execution life-cycle of the thread, it mayreference, at 340, some specific portions of the loaded data within thecache store. A reference may be made with a variety of API operations orcalls for such things as, by way of example only, getting, flushing,refreshing, reading, writing, and/or updating. The API operations orcalls include parameters associated with a reference to the thread localcache and keys. The keys are symmetrical between the thread and thethread local cache service because both them used the same keydefinitions to acquire the keys. The keys possessed by the thread localcache service provide locations within an index of the cache store andthe entry values identified at those locations includes references theactual locations within the cache store where the desired data resides.

For example, at 340, the thread may utilize API calls or operations tomodify some portions of the loaded data or to update data sourcesassociated with some portions of the loaded data housed and managed bythe thread local cache service via the thread local cache reference. Todo this, the thread local cache service utilizes the reference to thethread local cache to identify the specific cache store and the keysinto index locations. Armed with this information, the thread localcache can use the keys to acquire the desired portions of data forpurposes of retrieving, flushing, writing new data, updating theexisting data to one or more data stores, etc.

It should be noted that the thread local cache service may manage aplurality of thread local caches for a variety of threads. In a similarmanner, the custom-caching service and the key service of the FIGS. 1and 2 may also manage a plurality of thread local caches for a varietyof threads and/or a plurality of global caches for a variety ofapplications.

FIG. 4 is a diagram of a custom cache system 400, according to anexample embodiment. The custom cache system 400 is implemented in amachine-accessible and readable medium and is accessible over a network.The network may be wired, wireless, or combinations of wired andwireless. In an embodiment, the custom cache system 400 implementswithin its components, among other things, the custom-cache service, thekey-service, and the thread local cache service associated with themethods 100, 200, and 300 of FIGS. 1-3, respectively.

The custom cache system 400 abstracts cache stores (memory heap space)to a series of higher-level references referred to as caches. An APIassociated with the caches permit applications or threads of anapplication to custom control and insert, remove, modify, and/or writedata having a variety of data types to and from the cache stores viareferences to the caches and via calls to methods or operations of theAPI. The backend processing of the API calls drive some of theprocessing associated with the custom cache system 400. Other processingof the custom cache system 400 may be driven by policies oradministrator interfaces communicating with the thread local cacheservice.

The components of the custom cache system 400 are presented for purposesof illustration only and are therefore not intended to limit embodimentsof the custom cache system 400. Accordingly, some components may beomitted, some other components may be added, and some collections of thecomponents may be combined into single components. All such arrangementsare intended to fall within the scope of the embodiments presented here,if such arrangements provide for the novel techniques for custom cachingas described herein.

In one implementation, the custom cache system 400 is implemented as anOO system having an OO API, which is accessible to the applications andthe threads of an application. The objects may be defined within aprogramming language as classes having public methods accessible to theapplications and threads and protected methods accessible to certainobjects defined within the custom cache system 400. The classes mayinherit from one another and may permit polymorphism. Of course it isrecognized that any software implementation of the custom cache system400 is intended to fall within the scope of the embodiments presentedherein, thus the custom cache system 400 may also be implemented inother manners, such as a series of services defined with traditionalstructured programming, and the like.

The custom cache system 400 includes a key manager 401A, key definitions401B, a global cache manager 402B, a thread local cache manager 402C, acache entry loader 403A, a cache store 403B, and a cache loader 404. Insome embodiments, the custom cache system 400 also includes a cachemaintainer 405, a cache eviction policy 406, and/or a cache statisticcollector 407. The operations and features of the custom cache system400 operate on and through instances of caches 402A. Each of these willnow be discussed in turn.

The key manager 401A is responsible for managing key definitions 401B.The key manager 401A and the key definitions 401B and configured and orcreated through API operations processed by an application or thread inconnection with an instance of a cache 402A. The key definitions 401Bknow how to parse different types of data or data objects to acquirevalues to generate keys. Each key definition 401B is associated with aspecific key type and when supplied a value for data associated with aspecific key type generates a symmetric key. The key is symmetricbecause both the application and the cache entry loader 403A can produceidentical keys when using the same key type and the same value. Some keydefinitions may permit the creation of keys associated with logicalcollections of data. Techniques for achieving this were discussed indetail above.

The cache managers 402B and 402C create and manage instances of caches402A with cache stores 403B on behalf of the application threads. Thecache managers 402B and 402C return references to created cacheinstances 402A to the application threads. The application threads usethe cache instance 402A reference to perform operations via its API. Acache instance 402A may be a global cache that is shared between threadsor a cache may be a thread local cache that is exclusively associatedand accessed by a particular thread. The cache managers 402B and 402Cinteract with the other components of the custom cache system 400 viathe references to specific cache instances 402A for purposes of managingthe cache stores 403B and maintaining their integrity.

The cache entry loader 403A is provided references to the key manager401A and cache loader 404 by the cache instance 402A, through which itaccesses the key manager 401A and its key definitions 401B that itcontains. The cache entry loader 403A holds the reference to the cachestore 403B. The cache loader 404 is responsible for determining whereand how to acquire data from a variety of data sources, where the datais potentially associated with a variety of disparate data types orobjects. Thus, the data access layer is abstracted and hidden from theapplication threads. This makes the application logic that interactswith the cache instance 402A more portable and less coupled to thespecific data sources and their methods for acquiring data, such thatshould a data source change its interface the application threads wouldstill be operational because it is the cache loader 404 that managesdata stores and their interfaces.

The cache loader 404 may cause all data identified by an application tobe loaded into the cache store 403B when initially called, such as whenthe cache instance 402A referenced by the application thread is a globalshared cache. The cache instance 402A makes this happen via a call tothe cache entry loader 403A; and the cache entry loader 403A interactsdirectly with the cache loader 404 to obtain data that the cache loader404 determines should be placed in the cache store 403B; and then thecache entry loader 403A subsequently loads the data into the cache store403B. Alternatively, data may be loaded into the cache store 403Binteractively or in a piecemeal fashion as instructed by the applicationthread via API calls on the cache instance's 402A public interfaceissued by the application thread, such as when the cache instance 402Areferenced by the application thread is a thread local cache or is alazy loaded global cache.

The cache loader 404 acquires data from the data sources associated withthe data types or objects and causes units, records, or portions of thedata acquired from the data sources to be indirectly passed on to thecache entry loader 403.

The cache entry loader 403A interacts with the key manager 401A toacquire symmetric keys for data being loaded. The keys identifylocations within an index being maintained for the cache store 403B.Each location has a reference to specific data loaded in the cache store403B. In some instances, a location may include a pointer to a datastructure, where the data structure includes references to multiple datawithin the cache store 403B. This is done when a key definition 401Bprovided collection keys for some of the data, as was discussed above indetail.

The cache instances 402A interacts with API calls initiated by theapplication threads to drive the loading, retrieval, and otherpotentially volatile operations (e.g., writing, flushing, refreshing,etc.) against the data included within the cache stores 403B. Thevolatile operations by default are constrained on global cache instances402A and not on thread local cache instances 402A and are enforced bythe cache instances 402A should one of these operations be attempted byan application thread against a global shared cache type with thisdefault configuration. The operations initiated by the applicationthreads from the API can include a variety of parameters, such as areference to a cache, a key, a key manager, and a data object. Moreover,if the operation dictates, the cache instance 402A may also consume akey for purposes of looking up in the cache store 403B a specific pieceof data identified by the application thread via the key. Theseoperations are driven via the cache instances' 402A interfaces.

In an embodiment, the custom cache system 400 may also include a cachemaintainer 405. The cache maintainer 405 interacts directly with thecache instance 402A and is responsible for managing the state of thedata in the cache instance 402A via configured refresh operations onglobal caches 402A. The cache maintainer 405 will initiate a refreshsequence using a background thread based on configuration, such as time.During this refresh sequence it will call the cache instance 402A, whichwill subsequently call the cache loader 404, which will determine if andwhat portion of the cache should be flushed and/or updated. The cacheinstance 402A will then create a new cache entry loader 403A instanceand cache store 403B instance and begin to populate it with the dataprovided by the cache loader 404. Once the new cache store 403B has beensuccessfully loaded, the old cache entry loader 403A reference will bereplaced with a reference to the new cache entry loader 403A instancecontaining the new cache store 403B. This approach to refresh allows forthe application threads to continue to use the cache while the new cacheentry loader 403A and cache store 403B are created, and it allows forthese objects to be instantaneously replaced; thus, allowing foruninterrupted access to cache data for the application threads evenduring a refresh operation. Again, a global cache (also known as ashared cache) is a cache instance 402A with an associated cache store403B, where data indexed therein is shared between pluralities ofthreads of a same application.

In still another embodiment, the custom cache system 400 may alsoinclude a cache eviction policy 406. The cache eviction policy may beconsumed and evaluated by the cache instance 402A of the cache store403B for purposes of determining the conditions and/or events underwhich portions of the data within the cache stores 403B are to beremoved from the cache stores 403B and/or updated to one or more datasources associated with the portions of the data. In some cases, thecache eviction policy 406 may be an electronic data structure or file,such as an extensible markup language (XML) formatted file. In othercases, the cache eviction policy 406 is dynamically communicated from aglobal administrator service.

In still another embodiment, the custom cache system 400 may alsoinclude a cache statistics collector 407. The cache statistics collector407 collects statistical data from the caches regarding usage andefficiency and reports the same to a configurable statistic collectionservice.

FIG. 5 is a diagram of another custom cache system 500, according to anexample embodiment. The custom cache system 500 is implemented in amachine-accessible and readable medium and is accessible over a network.The network maybe wired, wireless, or a combination of wired andwireless. The custom cache system 500 provides an alternative view ofthe custom cache system 400 of FIG. 4.

The custom cache system 500 includes an API 501 and a cache creator 502.In an embodiment, the custom cache system 500 may also include a keyacquirer 503 and/or a cache synchronizer 504. Each of these will now beaddressed in turn.

The API 501 includes a suite of definitions, operations, calls, ormethods which are available to a developer to include in his/her codeassociated with an application or a thread of an application. Thedefinitions define abstracted views of cache stores via cachedefinitions. Each cache definition includes a set of operations forperforming some operation against the cache store via references to acache.

A cache can be of two types: a global shared cache and a thread localcache. Other operations may permit cache configurations to acquire dataof application-defined types from a plurality of data sources. Stillmore operations may permit the communication of key managers associatedwith key definitions to be communicated. Some key definitions may beassociated with collective keys. In yet other operations, gets, puts,sets, writes, refreshes, flushes, updates, and the like permit volatileand non volatile operations to be performed against data housed in thecache stores and referenced by the cache.

The front-end of the API 501 interfaces with applications or threads ofan application. Conversely, the back-end of the API 501 interfaces orcommunicates with the cache creator 502 and optionally the key acquirer503 and/or the cache synchronizer 504.

The cache creator 502 handles API 501 operations executed by theapplication or threads of the application. In an embodiment, the cachecreator 502 is implemented within software on a machine-accessible andreadable medium as a means for creating and managing customized caches.Examples of objects, services, and systems for achieving this werediscussed above with respect to the methods 100, 200, and 300 and withrespect to the system 400. Accordingly, these descriptions associatedwith these methods 100-300 and the system 400 are incorporated byreference herein as the means for creating and managing customizedcaches. Some customized caches are shared between the same threads of anapplication, while other ones of the customized caches are managed byspecific threads to the exclusion of other ones of the threads.

In an embodiment, the some features of the cache creator 502 may bedelegated to a key acquirer 503. The key acquirer 503 manages keys forthe customized caches. In an embodiment, the key acquirer 503 isimplemented in software within a machine-accessible and readable mediumas a means for acquiring keys. The technique and features of managingand acquiring keys for customized caches where presented above withrespect to the methods 100-300 and the system 400. Accordingly, themeans for acquiring keys includes the descriptions and techniquespresented above with respect to key creation, key definitions, and keymanagement of the methods 100-300 and the system 400. In an embodiment,the means for acquiring keys 503 provides a symmetrical key between boththe application or threads and the customized caches.

In yet another embodiment, the custom cache system 500 includes a cachesynchronizer 504. The cache synchronizer 504 permits data associatedwith disparate cache stores (disparate heaps) to be synchronized withone another. In an embodiment, the cache synchronizer 504 is implementedin software within a machine-accessible and readable medium as a meansfor synchronizing data between numbers of disparate customized cachesvia their corresponding cache stores. The means for synchronizing datamay be defined within policies communicated to the affected cachestores. Alternatively, the means for synchronizing data may bedynamically communicated by a global administrator service at predefinedor dynamically instructed intervals.

It has now been demonstrated how custom caching can be achieved forpurposes of improving processing throughput of an application andstreamlining development time associated with developing theapplication. Additionally, the techniques presented herein also reducethe number of data store transactions that may be needed to process anapplication to completion. Thus, it is apparent that a significantreduction in data store transactions can be achieved when applicationsthat access the data store employ the API and techniques presentedherein.

FIGS. 6-10 are now presented and described for purposes of illustratingexample environments and transactional architectures and arrangementswithin which the embodiments of this invention may be practiced. Theexample architectures and arrangements provide a few environments inwhich the embodiments of this invention may usefully be deployed;however, it is noted the embodiments presented above are not intended tobe limited or constrained by the specific examples presented below.

FIG. 6 is block diagram illustrating an example network-based commercesystem or facility 600 wherein embodiments of the present invention maybe deployed. While an example embodiment of the present invention isdescribed within the context of the network-based commerce system 600,the invention will find application in many different types ofcomputer-based, and network-based, facilities (commerce, transaction orotherwise).

The network-based commerce system 600 includes one or more of a numberof types of front-end servers that may each include at least one DynamicLink Library (DLL) to provide selected functionality. The system 600 mayinclude page servers 612 that deliver web pages (e.g., mark-up languagedocuments), picture servers 614 that dynamically deliver images to bedisplayed within Web pages, listing servers 616 that facilitatecategory-based browsing of listings, search servers 618 that handlesearch requests to the system 600 and facilitate keyword-based browsingof listings, and ISAPI servers 620 that provide an intelligent interfaceto a back-end of the system 600. The system 600 may also include e-mailservers 622 that provide, among other things, automated e-mailcommunications to users of the network-based commerce system 600.

In an embodiment, one or more administrative application functions 624facilitate monitoring, maintaining, and managing the system 600. One ormore API servers 626 may provide a set of API functions for querying andwriting to the network-based commerce system 600. APIs may be calledthrough the HTTP transport protocol. In an embodiment, information issent and received using a standard XML data format. Applicationsutilized to interact (e.g., upload transaction listings, reviewtransaction listings, manage transaction listings, etc.) with thenetwork-based commerce system 600 may be designed to use the APIs. Suchapplications may be in an HTML form or be a CGI program written in C++,Perl, Pascal, or any other programming language.

The API servers 626, page servers 612, picture servers 614, ISAPIservers 620, search servers 618, e-mail servers 622 and a databaseengine server (e.g., provided by one or more of the application servers628) may individually, or in combination, act as a communication engineto facilitate communications between, for example, a client machine 630and the network-based commerce system 600; act as a transaction engineto facilitate transactions between, for example, the client machine 630and the network-based commerce system 600; and act as a display engineto facilitate the display of listings on, for example, the clientmachine 630.

The back-end servers may include a database engine server, a searchindex server and a credit card database server, each of which maintainsand facilitates access to a respective database.

In an embodiment, the network-based commerce system 600 is accessed by aclient program, such as for example a browser 636 (e.g., the InternetExplorer distributed by Microsoft Corp. of Redmond, Wash.) that executeson the client machine 630 and accesses the network-based commerce system600 via a network such as, for example, the Internet 638. Other examplesof networks that a client may utilize to access the network-basedcommerce system 600 include a wide area network (WAN), a local areanetwork (LAN), a wireless network (e.g., a cellular network), the PublicSwitched Telephone Network (PSTN) network, or the like. The clientprogram that executes on the client machine 630 may also communicatewith the network-based commerce system 600 via the API servers 626.

Example Database Structure

FIG. 7 is a database diagram illustrating an example database 740,maintained by and accessed via the database engine server 728, which atleast partially implements and supports the network-based commercesystem 700. In an embodiment a database engine server may maintain aplurality of databases 740. For example, may maintain a master writedatabase 741 (e.g., including a plurality of horizontally distributeddatabases), and a read-only database 743 that may, for example, allowloads to be balanced appropriately.

The database 740 may, in an embodiment, be implemented as a relationaldatabase, and includes a number of tables having entries, or records,that are linked by indices and keys. In an alternative embodiment, thedatabase 740 may be implemented as collection of objects in anobject-oriented database, as discussed by way of example in more detailbelow.

The database 740 includes a user table 742 that contains a record foreach user of the network-based commerce system 700. An example recordfor each user is shown in FIG. 9. A user may operate as a seller, abuyer, or both, when utilizing the network-based commerce system 700.The database 840 also includes listings tables 744 (see FIGS. 8 and 9)that may be linked to the user table 742. The listings tables 744 mayinclude a seller listings table 746 and a bidder listings table 748. Auser record in the user table 742 may be linked to multiple listingsthat are being, or have been, listed or offered for sale via thenetwork-based commerce system 700. In an embodiment, a link indicateswhether the user is a seller or a bidder (or buyer) with respect tolistings for which records exist within the listings tables 744.

The database 740 also includes one or more divisions in the form ofcategories provided in category tables 750. Each record within thecategory table 750 may describe a respective category. In an embodiment,listings provided by the system 700 are arranged in the categories.These categories may be navigable by a user of the network-basedcommerce system 700 to locate listings in specific categories. Thus,categories provide a mechanism to locate listings that may be browsed.In addition or instead, an alphanumeric search mechanism may be providedby the search servers 718 to allow a user to search for specificlistings using search terms or phrases.

In an embodiment, the category table 750 describes multiple,hierarchical category data structures, and includes multiple categoryrecords, each of which describes the context of a particular categorywithin the multiple hierarchical category structures. For example, thecategory table 750 may describe a number of real, or actual, categoriesto which listing records, within the listings tables 744, may be linked.

The database 740 is also shown to include one or more attributes tables752. Each record within the attributes table 752 describes a respectiveattribute associated with a listing. In an embodiment, the attributestable 752 describes multiple, hierarchical attribute data structures,and includes multiple attribute records, each of which describes thecontext of a particular attribute within the multiple hierarchicalattribute structures. For example, the attributes table 752 may describea number of real, or actual, attributes to which listing records, withinthe listings tables 744, may be linked. Also, the attributes table 752may describe a number of real, or actual, attributes to whichcategories, within the category table 750, may be linked.

The database 740 may also include a note table 754 populated with noterecords that may be linked to one or more listing records within thelistings tables 744 and/or to one or more user records within the usertable 742. Each note record within the note table 754 may include, amongother things, a comment, description, history or other informationpertaining to a listing being offered via the network-based commercesystem 700, to a user of the network-based commerce system 700. Thedatabase 740 may also include a targeted site table 756 populated withtargeted site records that may be linked to one or more listing recordswithin the listings tables 744 and/or to one or more user records withinthe user table 742.

A number of other example tables may also be linked to the user table742, namely a user past aliases table 758, a feedback table 760, afeedback details table 762, a bids table 764, an accounts table 766, andan account balances table 768. In an embodiment, the database 740 alsoincludes a batch table 770, a batch listings table 772, and a listingswait table 774. The data may be partitioned across multiple databaseinstances, and queries may have to be executed against multiple databaseinstances and query results may need to be aggregated.

FIG. 10 shows a diagrammatic representation of a machine in the exampleform of a computer system 1000 within which a set or sequence ofinstructions, for causing the machine to perform any one of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may comprise a network router, a networkswitch, a network bridge, Personal Digital Assistant (PDA), a cellulartelephone, a web appliance, set-top box (STB) or any machine capable ofexecuting a sequence of instructions that specify actions to be taken bythat machine.

The computer system 1000 includes a processor 1002, a main memory 1004and a static memory 1006, which communicate with each other via a bus1008. The computer system 1000 may further include a video display unit1010 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)).The computer system 1000 also includes an alphanumeric input device 1012(e.g., a keyboard), a cursor control device 1014 (e.g., a mouse), a diskdrive unit 1016, a signal generation device 1018 (e.g., a speaker) and anetwork interface device 1020 to interface the computer system to anetwork 1022.

The disk drive unit 1016 includes a machine-readable medium 1024 onwhich is stored a set of instructions or software 1026 embodying anyone, or all, of the methodologies described herein. The software 1026 isalso shown to reside, completely or at least partially, within the mainmemory 1004 and/or within the processor 1002. The software 1026 mayfurther be transmitted or received via the network interface device1020.

For the purposes of this specification, the term “machine-readablemedium” shall be taken to include any medium which is capable of storingor encoding a sequence of instructions for execution by the machine andthat cause the machine to perform any one of the methodologies of thepresent invention. The term “machine-readable medium” shall accordinglybe taken to included, but not be limited to, solid-state memories,optical and magnetic disks, and carrier wave signals. Further, while thesoftware is shown in FIG. 10 to reside within a single device, it willbe appreciated that the software 1026 could be distributed acrossmultiple machines or storage media, which may include themachine-readable medium.

It should also be noted that in some embodiment the methods 100, 200,and 300 may be implemented on a machine-accessible medium havinginstructions. The instructions when executed by a machine perform themethods 100, 200, and 300. The medium may be removable media that isinterfaced to the machine. Alternatively, the medium may be fixed mediathat is an integral part of the machine. In this manner, theinstructions of the methods 100, 200, and 300 may be pressed onremovable media and uploaded to a machine, preloaded within a machine,downloaded on a machine from a peripheral device, or downloaded from aremote machine over a network to a different machine.

The above description is illustrative, and not restrictive. Many otherembodiments will be apparent to those of ordinary skill in the art uponreviewing the above description. The scope of embodiments shouldtherefore be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

The Abstract is provided to comply with 37 C.F.R. §1.72(b) and willallow the reader to quickly ascertain the nature and gist of thetechnical disclosure. It is submitted with the understanding that itwill not be used to interpret or limit the scope or meaning of theclaims.

In the foregoing description of the embodiments, various features aregrouped together in a single embodiment for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting that the claimed embodiments have more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate example embodiment.

1. (canceled)
 2. A system for customized caching, comprising: aprocessor; and a storage medium having instructions stored thereon thatare executable to cause the system to perform operations comprising:receiving, via an application programming interface (API) call from aclient application, a request to create a customized cache instance forthe client application; acquiring configuration information, includingkey manager information, for the customized cache instance; establishingthe customized cache instance using the configuration information;returning, to the client application, a cache reference to thecustomized cache instance; receiving, based on the cache reference, adata request from the client application to access requested data withinthe customized cache instance, the data request including anapplication-generated key that is usable to determine a location of therequested data within a data store of the customized cache instance;locating the requested data using the application-generated key; andreturning, to the client application, the requested data.
 3. The systemof claim 2, wherein the client application is a multi-threadedapplication, and wherein the request is received from a particularthread of the multi-threaded application.
 4. The system of claim 2,wherein the configuration information includes an identifier for a datasource and a mechanism for acquiring data from the data source.
 5. Thesystem of claim 2, wherein the customized cache instance is a cache thatincludes an abstracted and encapsulated version of a cache store.
 6. Thesystem of claim 2, wherein the operations further comprise: establishingthe customized cache instance as a thread local-only cache for aparticular thread of the client application, wherein the threadlocal-only cache is not accessible by other threads.
 7. The system ofclaim 2, wherein the operations further comprise: establishing thecustomized cache instance as a global cache for the client application,wherein data in the global cache is accessible by a plurality of threadsof the client application using the API.
 8. The system of claim 2,wherein data in the data store is not directly accessible by the clientapplication, and instead requires use of the API for access by theclient application.
 9. A method, comprising: receiving, at customcaching computer system via an application programming interface (API)call from a client application, a request to create a customized cacheinstance for the client application; the caching computer systemreceiving configuration information for the customized cache instancefrom the client application; establishing the customized cache instanceusing the configuration information; returning, to the clientapplication, a cache reference to the customized cache instance;receiving, via the API, a data request from the client application toaccess requested data within the customized cache instance, the datarequest including a key generated by the client application that isusable to determine a location of the requested data within a data storeof the customized cache instance; locating the requested data using thekey; and returning, to the client application, the requested data. 10.The method of claim 9, wherein establishing the customized cacheinstance includes storing a pointer to a key manager based on keymanager information included in the configuration information.
 11. Themethod of claim 10, wherein the key manager includes informationspecifying a plurality of key definitions, each usable to locate anidentifier within a specified type of data object to uniquely identifyinstances of that type of data object.
 12. The method of claim 9,further comprising evicting data from the data store in accordance witha cache eviction policy.
 13. The method of claim 9, wherein thecustomized cache instance is a local instance accessible only by aparticular thread of a plurality of threads of the client application.14. The method of claim 9, wherein the customized cache instance is aglobal instance accessible by a plurality of threads of the clientapplication.
 15. The method of claim 9, further comprising acquiring, bythe custom caching system, a plurality of data objects for thecustomized cache instance via a plurality of disparate data sources. 16.The method of claim 9, wherein the configuration information includeskey manager information comprising a pointer to a key manager having oneor more key definitions.
 17. The method of claim 9, wherein the clientapplication is running on the custom caching computer system.
 18. Amethod, comprising: making an application programming interface (API)call, by a client application running on a computer system, to requestcreation of a customized cache instance for the client application;passing configuration information, by the client application, to acustom caching service, wherein the configuration information includeskey manager information; receiving, at the client application, anindication that the customized cache instance has been created by thecaching service, wherein data within the customized cache instance isnot directly accessible by the client application but is accessibly bythe client application via the API; making a data request to the cachingservice, by the client application via the API, to access requested datawithin the customized cache instance, the data request including a keythat is usable to determine a location of the requested data within adata store of the customized cache instance; and the client applicationreceiving the requested data from the caching service.
 19. The method ofclaim 18, wherein the customized cache instance is a local cacheinstance accessibly only by a particular thread of a plurality ofthreads of the client application.
 20. The method of claim 18, whereinthe key manager information includes information specifying a pluralityof key definitions, each usable to locate an identifier within aspecified type of data object to uniquely identify instances of thattype of data object.